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Abstract 



Student Experiences with Information Technology 
and their Relationship to Other Aspects of Student Engagement 

Considerable efforts have been made to get students to use information technology for 
educational purposes. This study uses data from the 2003 administration of the National Survey 
of Student Engagement (NSSE) to investigate the relationships between student uses of 
information technology and other forms of student engagement. The results suggest that there is 
a strong positive relationship between using information technology for educational purposes 
and other effective educational practices such as active and collaborative learning and student- 
faculty interaction. The results also raise questions about whether engagement with information 
technology should be viewed as a form of engagement in and of itself. Perhaps when students 
use information technology it increases the opportunities for other types of engagement. 
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Student Experiences with Information Technology 
and their Relationship to Other Aspects of Student Engagement 

On college campuses across the country, students, faculty, and administrators are using 
computers, the internet, and other forms of information technology for various educational 
purposes. No longer are email, the World Wide Web (WWW), and word processing flashy new 
tools used by a select few. Rather, using these tools is now as commonplace as using a telephone 
or backpack. It appears that we have, indeed, reached a point where certain applications of 
information technology are as ubiquitous as many just a decade ago predicted (Dolence & 

Norris, 1995; Green, 1996). 

The results from studies of student information technology use for academic purposes are 
promising. Eor example. The Institute for Higher Education Policy (1999) reported that using 
email for academic work grew from 8% in 1994 to 44% by 1998. The percentage of courses 
using the internet doubled from 15% in 1996 to 30% by 1998. A more recent national survey 
found that 84% of college students owned a computer and that 99% used the internet, with 66% 
doing so daily (Student Monitor, 2003). Students appear to use the internet to communicate with 
others and to find materials and assistance with their coursework (Hu & Kuh, 2001; Student 
Monitor, 2003). 

Given the relative speed with which computers and the internet have made their way into 
higher education, many question whether these technologies offer educational benefits. The 
limited evidence to date, mostly from classroom-level investigations, suggests that students’ use 
of information technology has a positive effect on several important educational outcomes. Eor 
example, a study of course redesign projects at several institutions suggests that the incorporation 



of technology into a course results in greater learning, measured in various ways, for students in 
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the redesigned course compared to similar courses without the redesign (Twigg, 2004). 

Similarly, the few studies on general college populations (i.e., not at the classroom-level) 
indicate that student use of information technology positively affects a variety of outcomes such 
as student self-reported gains in general education, personal development, and intellectual 
development (Hu & Kuh, 2001; Kuh & Hu, 2001; Kuh & Vesper, 2001). 

However, some studies show mixed results. For example. Flowers, Pascarella, and Pierson 
(2000) found that using information technology positively influenced cognitive development for 
two-year college students, but not necessarily for students at four-year colleges. The researchers 
conjectured that the mixed findings may be associated with students from higher socioeconomic 
backgrounds being advantaged in terms of information technology use (Gladieux & Swail, 

1999). In addition, those with higher ability levels (Dillon & Gabbard, 1998) and those attending 
institutions that are more “wired” (Hu & Kuh, 2001) are also more likely to use and benefit from 
information technology. 

Differential use of information technology by different types of students has led to 
questions about the possible negative effects of information technology. For example, Reisberg 
(2000) suggests that uses of information technology may distract students from participating in 
empirically confirmed effective educational practices. Arguably, certain technologies, such as 
gaming machines like Nintendo or Xbox, and applications such as downloading music may have 
few educational benefits. At the same time, there is evidence to suggest that educationally 
purposeful uses of information technology, such as emailing faculty members or other students 
about assignments, can promote collaboration among students (Alavi, 1994; Oblinger & 
Maruyama, 1996) as well as foster more frequent contacts between students and faculty (Hu & 



Kuh, 2001; Kuh & Hu, 2001; Wingard, 2004). 
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If using information technology is potentially an effective educational practice, what is its 
relationship to other such practices? Kuh and Hu (2001) suggest that using information 
technology has a strong positive relationship with an overall measure of student engagement. In 
a study of “best wired campuses” (institutions that have made large investments in technology), 
students report slightly more frequent contact with faculty and participate more in active learning 
activities compared with their counterparts attending less wired campuses (Hu & Kuh, 2001). 
These results point to a positive link between information technology use and engagement in 
effective educational practices. 

Most of the extant research treats student use of information technology as a separate form 
of student engagement. However, as Kennedy (2000) suggests, perhaps information technology 
is an avenue for students to increase their engagement in areas such as active and collaborative 
learning or student-faculty interaction. “Used appropriately and in concert with powerful 
pedagogical approaches, technology is supposed to enhance student learning productivity” (Kuh 
& Vesper, 2001, p. 87). There is evidence that this is the case (e.g., Kuh & Hu, 2001; Twigg, 
2004), but it is not clear, especially now that technology use is so commonplace, whether we 
should be treating student uses of information technology as separate from other forms of 
engagement or as one of several avenues to promote or enhance other forms of student 
engagement with effective educational practices. 

Purpose of the Study 

This study expands our understanding of the relationships between students’ uses of 
information technology and their involvement in other educationally effective practices that prior 
research shows lead to desired educational outcomes. Our exploration proceeds along two paths. 
The objective of the first is to develop a scale that measures student use of information 
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technology and to test the direction and strength of its relationships with other established 
indicators of student engagement. The objective of the second path is to examine the merits of 
shifting our perspective to view particular uses of information technology as a component of 
other forms of student engagement rather than a form in and of itself. That is, we seek to 
determine whether engagement with information technology is its own form of engagement or 
whether information technology is a mechanism through which students engage is existing 
effective educational practices. For example, emailing faculty about academic matters has been 
treated as a way that students use information technology for educationally relevant purposes. 
However, from another perspective, we could conceptualize use of email as a way for students to 
create more opportunities for interacting with their instructors. 

Methods 

Data Source 

The data for this study come from the 2003 administration of the National Survey of 
Student Engagement (NSSE), an annual survey of college students at four-year institutions that 
measures students’ participation in educational experiences that prior research has connected to 
valued outcomes (Chickering & Gamson, 1987; Kuh, 2001a, 2003). About 350,000 first-year 
students and seniors were randomly selected from files provided by the 437 participating 
colleges and universities. The NSSE standard sampling scheme calls for an equal number of 
first-year and senior students to be selected with the size determined by the number of 
undergraduate students enrolled at the institution. Students at about three-quarters of the 
institutions had the option of responding either via a traditional paper questionnaire or online. 
About a quarter of the colleges and universities opted to administer only online (students 
received an introduction letter through the mail and all further contact was online). 
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Response rates at the participating institutions ranged from 7% to 78% with an average 
institutional response rate for NSSE 2003 of 43%. The response rates at paper schools 
(institutions where students had the option of completing either the paper or the Web version of 
the survey) were similar to those of Web-only schools (institutions where students only had the 
option of completing the survey online) with averages of 43% and 44%, respectively. About 
48% of the respondents completed the paper version of the survey and approximately 52% 
completed it using the Web. 

The survey itself, The College Student Report, focuses on how much students participate in 
effective educational practices. Eor example, students are asked to identify how often they make 
class presentations, participate in a community-based project as a part of a course, and work with 
faculty members on activities other than coursework. In addition, students identify the degree to 
which their courses emphasize different mental processes (e.g., memorizing and synthesizing), 
how many hours per week they spend studying, working, or participating in co-curricular 
activities, as well as how they would characterize their relationships with people on campus (to 
see the survey visit the NSSE website, www.iub.edu/~nsse). 

Because the NSSE survey is intentionally short, it is not possible to examine every 
worthwhile topic in depth. In addition, the behaviors and conditions represented by the survey 
items were selected because of their demonstrated relationship to desired learning outcomes. 
Because the research literature linking information technology and learning is sparse, few of the 
survey questions address this area. Eor example, only two items on the core survey in 2003 
focus on the frequency with which students utilize information technology (i.e., how often have 
you used an electronic medium to discuss or complete an assignment and how often have you 
used e-mail to communicate with an instructor). 
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Each year, NSSE tests new survey items. In 2003, based on inquiries from several groups 
(e.g., librarians) and from large universities, a series of items about students’ experiences with 
information technology were added to the end of the online survey to further investigate 
students’ experiences with information technology and test items for possible inclusion in future 
administrations. 

Measures 

The technology items in Table 1 were developed over a few months. To identify behaviors 
and activities related to information technology worth exploring, we sought advice from several 
experts. Based on their suggestions and the available literature we developed a series of 
questions that represented a wide range of information technology uses that had the potential to 
enhance student learning as well as some that could conceivably be counterproductive, such as 
Game Boys. Erom a pool of about 30 items we selected about 20 and circulated them to our 
expert panel. After receiving their feedback and making final revisions we ultimately decided to 
include 18 on the 2003 online version of NSSE. The experimental information technology items 
that are the focus of this study include questions about the frequency with which students use 
information technology (e.g., computers and the WWW) for academic and non-academic 
purposes, the amount of time students spend online, and the amount their instructors use and 
require the use of information technology (see Table 1 for the complete list of items tested in 
2003). 

Beyond the technology items, two other groups of measures are used in this study. Each 
year, NSSE calculates national and institutional performance on five clusters of effective 
educational practice (Kuh, 2001a, 2003). They are: academic challenge, active and collaborative 
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learning, student-faculty interaction, enriching educational experiences, and supportive campus 
environment. 

Academic challenge measures the amount and difficulty of the academic work required of 
students, the amount of time they devote to preparing for class, and their perceptions of the 
degree to which the institution emphasizes academic achievement. Active and collaborative 
learning is a measure of the frequency with which students work independently or with peers to 
solve problems, inside and outside the classroom and participate in other activities that 
encourage them to connect their learning to real world problems (e.g., community service). 
Student-faculty interaction reflects the frequency with which students interact or work with their 
instructors in and outside of class. Enriching educational experiences measures students’ 
participation in a wide array of activities that have educationally beneficial effects on many 
students. Supportive campus environment is a measure of the degree to which students’ perceive 
that their institution supports their academic and social needs and the quality of relations among 
different groups on campus (other students, faculty, administrators). 

The scores for each of the five clusters, commonly known as benchmarks of effective 
educational practice, are based on students’ responses to 42 core survey items that are grouped to 
measure student engagement in each of these areas. Using the item groupings (see National 
Survey of Student Engagement, 2003), student-level scales that parallel the benchmarks were 
created for this study. The reliability of each scale is reported in Table 2. 

Additionally, the analyses will use the 22 items from the first question on the core survey. 
Many of these items are used in the construction of the scales described above. The question 
stem and response categories for these items are identical to those of most of the technology 
items and consequently these items provide the base for an exploratory factor analysis (described 
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below) that examines whether engagement with information technology is a stand alone concept 
or a part of other forms of engagement. 

Sample 

The sample for this study consists of over 60,000 students from more than 420 four-year 
colleges and universities across the country. Out of the total sample, approximately 60% are 
female, 74% are white (5% African American, 6% Asian, 4% Hispanic, 1% Native American, 
and 11% other racial/ethnic background), and 56% are first-year students. In addition, 16% have 
transferred from another institution, 72% live on campus, and 94% are full-time students. 

All of the students completed the online version of the NSSE survey including the 
experimental information technology questions. Typically, online completers differ from those 
students who fill out the paper survey. Eor example, a larger percentage of women (70%) and 
students of certain racial/ethnic groups (African American, Eatino/a, and American Indian) fill 
out the paper version of the survey. Also, paper completers are more likely to be older, part- 
time, live off campus, have parents with less formal education, and have transferred from a 
different institution. These differences might explain why paper completers score slightly lower 
on four out of the five established indicators of student engagement than those that fill the survey 
out online. 

Analyses 

As mentioned earlier, we explore the relationships between student engagement with 
information technology and other indicators of student engagement along two distinct lines. 

Eirst, we examine, at the item level, how students’ uses of information technology relate to other 
aspects of engagement. Additionally, factor analysis on the technology items is used to derive 
information technology engagement scales. Several of the technology items are not used in this 
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factor analysis, which like prior factor analyses using NSSE data (e.g., Kuh, 2001b) employs 
principle components extraction with an oblique rotation (Promax). Item 2i (see Table 1) was 
dropped because it is about librarians more than about information technology. Items 4 and 5 
were dropped due to the lack of variability in these items and the greater amount of missing 
responses. Items 6a and b were dropped because their scale of measurement was different and 
the nature of the questions differs markedly from that of the other items (if left in the analysis, 
these two items form their own factor). Item 7 was dropped due to the marked difference in the 
content of the question from those included. 

The relationships between the engagement with information technology scales and the five 
established NSSE scales are explored using partial correlations that control for student 
background characteristics (e.g., gender, race, and parental education) and several aspects of 
their collegiate experience (e.g., transfer status and membership in a social fraternity or sorority). 
In creating the technology scales in this way and correlating them with the other scales, we have 
assumed that engagement with information technology is a separate, but related, form of student 
engagement. 

In our second line of inquiry we abandon the assumption that engagement with information 
technology is a separate construct and explore how the factor structure of question 1 from the 
core survey changes with the inclusion of several of the technology items. Eour technology 
items (2a, b, c, and d) were selected for inclusion in this factor analysis. Given that question 1 
from the core survey had 22 items, we made an effort to keep the number of technology items 
added at around 4 or 5, the approximate number of items per factor that comes from an analysis 
on the question 1 items alone. The top 4 loading items from the one-factor solution for the 
technology item factor analysis (see Table 5 in the results) were selected as they form a coherent 
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cluster of items that may hang together as a separate indicator of engagement with technology. 
As with the other factor analyses in this study and prior analyses using NSSE data (e.g., Kuh, 
2001b), those for this part of the analysis use principle components extraction with an oblique 
rotation. 



Results 

In general, students’ responses to the technology items suggest that many students use 
information technology regularly for personal and academic uses and to communicate with other 
students and instructors. Eor example, of the respondents to the technology items, 73% of first- 
year students and 69% of seniors spent more than 5 hours per week online for any reason; 
whereas almost two-fifths (38% and 39% for first-year students and seniors, respectively) spent 
more than 5 hours per week online doing academic work. Most students (first-year students, 
78%; seniors, 81%) report that instructors frequently (often or very often) require the use of 
information technology (e.g., WWW, internet, computer conferencing, etc.). Over half of all 
students (first-year students, 51%; seniors, 58%), frequently communicated with classmates 
online in order to complete academic work. In addition, 80% of first-year students and 86% of 
seniors frequently used the WWW to obtain resources while slightly fewer (71% of first-year 
students and 79% of seniors) made judgments about the quality of those resources. 

One troubling note is that a sizeable majority (87%) of all students say that their peers at 
least “sometimes” copy and paste information from the WWW or internet for reports/papers 
without citing the source. Between one quarter (25% of first-year students) and about three 
tenths (31% of seniors) said their peers did this frequently. 

Those students who frequently (often or very often) use information technology for 



classroom-related activities or assignments are more likely than their counterparts to report that 
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their courses frequently (“quite a bit” or “very much”) emphasize higher order thinking skills, a 
component of academic challenge. For example, of those students who frequently 
communicated with classmates online to complete academic work, 84% said their courses 
regularly emphasized applying theories or concepts to practical problems or in new situations 
compared to 70% for those who did not frequently communicate with classmates online. Those 
same students also were more likely to report more frequent interactions with faculty; that is, 

64% said they “frequently” discussed grades or assignments with an instructor compared with 
only 44% of those who infrequently communicated with classmates online. Similarly, of those 
students who frequently use their institutions’ library websites to obtain resources, 77% report 
that their courses emphasize synthesizing and organizing ideas, information, and experiences and 
43% report discussing career plans with faculty frequently. The comparable figures for students 
who infrequently use their institutions’ library websites are 63% and 27% respectively. 

In addition, students who report that their instructors either frequently require the use of 
information technology or frequently use it in class are more likely to report frequently working 
in groups outside of class. For example, of those students who report that their faculty frequently 
use information technology in class, 59% indicate that they frequently work in groups outside of 
class using the technology. This compares to 41% for those students who report that their 
faculty infrequently use information technology in class. It also appears that most students spend 
a portion of their time studying online. Of those students who spend more than 10 hours 
studying per week, 47% spend more than 5 hours and fully 98% spend at least an hour per week 



online for academic purposes. 
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Engagement with Information Technology Scale 

Table 4 presents three possible factor solutions from the analysis performed on a subset of 
the technology items (as described in the methods section, several items were not included in the 
analyses for reasons related to item content and missing data). The two and three factor 
solutions split the technology items into a group of classroom-related items and a set (or two) 
related to using information technology to find resources and communicate with faculty. The 
cross-loading of a few of items in these two solutions coupled with the relatively low reliability 
of the second or second and third groups of items led us to favor the single factor solution. 

When taken as a single group of items, the technology items create a scale that is reliable (a = 
0.83) and that we interpret as a general indicator of students’ engagement with information 
technology for academic purposes. 

By taking the four highest loading items (2a - d in Table 1) from the one factor solution in 
Table 4, we created a short version of the scale that is reliable at a comparable level to the large 
scale (a = 0.79), but with far fewer items. This short version of the scale contains only 
classroom-related items, which are the items used in the subsequent factor analysis with other 
core survey items. It is included in this section for comparison purposes. 

After controlling for student background characteristics (e.g., gender, race, and parents’ 
education) and several characteristics of students’ collegiate experience (e.g., transfer status and 
major), the partial correlations in Table 5 indicate that engagement with information technology 
is positively associated with the five aspects of student engagement derived from the core 
survey. It is most strongly associated with academic challenge, active and collaborative learning, 
and student-faculty interaction (partial correlations range from 0.41 to 0.48) and the relationships 
are approximately as strong as those that exist among the five other engagement scales. 
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The partial correlations between the short version of the scale and the other engagement 
scales are generally weaker than those for the long version. However, the relationship between 
the short version and active and collaborative learning is only slightly less and remains the 
strongest of the relationships. This result is not surprising given the content of the items and it 
foreshadows the results of the factor analysis that contains the items from the short version of the 
information technology scale and items from question 1 from the core survey, the question that 
contains the active and collaborative learning items. 

Factor Analysis with Technology and Core Survey Items 

Table 6 reports the results of two factor analyses. The first is an analysis of only those 
items from question 1 on the core survey. This analysis was performed to compare to previous 
analyses on the items (see Kuh, 2001b) as well as to provide a base for comparison to the second 
analysis which includes four of the experimental technology items. 

Without the technology items included, the factor structure remains largely the same as in 
previous analyses (see Kuh, 2001b). Four factors are extracted and, after rotation, define largely 
the same groups of items. Three items load differently in the analysis for this study than in the 
previous work. The core survey item about using email to communicate with an instructor (one 
of the two core items about information technology) previously loaded higher on the first factor, 
a factor largely defined by student- faculty interaction with a few items that can be viewed as 
active learning items (these load lowest on the factor). In this analysis, it loads slightly higher on 
the second factor, a factor defined by collaborative learning items, than on the first factor. The 
item about the frequency with which students participated in a community-based project does not 
load highly on any factor in the previous analysis or the first analysis for this study, but switches 
from loading on the second factor in the previous analysis to the first factor in this study. 




Engagement with Information Technology 17 



Similarly, the item about including diverse perspectives in class loads on factors three and four, 
but switches from loading higher on factor four, a collection of items about the amount of effort 
students put into their coursework, in the previous analysis to loading slightly higher on factor 
three, a factor largely about diverse interactions, in this analysis. 

When the four experimental technology items are added into the analysis, the factor 
structure is still largely the same as in the first analysis, something that remains the case even if 
three, five, or six factors are extracted. Since they are about students’ use of information 
technology to collaborate with others and give class presentations, it is mot surprising that the 
experimental technology items load with the collaborative learning factor in the second analysis. 
In fact, the experimental items become the highest loading items on that factor and the two 
lowest loading items from the first analysis end up loading higher on factor one. The result is a 
group of items that is squarely about student collaboration with peers for academic purposes. 

Interestingly, the collaboration group of items from the first analysis is only marginally 
reliable as a scale (a = 0.65). However, with the addition of the experimental technology items 
and the shifting of two items out of the grouping, the resulting scale is more reliable (a = 0.79), 
suggesting that this grouping more clearly identifies the concept underlying the items. 

Notably absent from this analysis is a single factor containing all of the technology-related 
items. In the four factor solution, the four experimental items and the core survey item about 
using an electronic medium to work on an assignment load on factor two, but the item about 
emailing an instructor loads highest on the student-faculty interaction factor. Even if the analysis 
allows for five factors, the experimental technology items all still load over 0.47 on the 
collaborative learning factor, while the two core tech items and one faculty interaction item split 
off to form a factor on which one of the experimental items also loads above 0.50. The four 
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factor solution is preferred over the five factor solution because that fifth factor did not seem 
distinct or meaningfully different from the first two factors. 

Limitations 

The primary limitations of this study relate to the sample and to the information 
technology items used in the study. Only students who completed the NSSE 2003 survey online 
were given the experimental technology items. It is possible that these students, many of whom 
chose to take the survey online, use information technology more frequently than those who 
filled out the paper version of the survey (Carini, Hayek, Kuh, Kennedy, & Ouimet, 2003). 
Because all of the data used in this study come from the online administration of NSSE 2003, it is 
not possible for us to test whether students who choose to use the web are more engaged with 
information technology. There is, however, some indication in the results that the two groups 
differ in their background characteristics and levels of engagement in other areas. Consequently, 
the results of this study might change slightly if we were able to repeat the analyses within the 
broader sample. Eor example, the correlations between the engagement with information 
technology scale and the other engagement scales might not be as strong in the entire NSSE 2003 
sample. 

The items used in the technology scale were not a priori developed for the purpose of 
exploratory factor analysis with other engagement items. Had they been developed with such a 
purpose in mind, different questions about student uses of information technology may have 
been included that would have consequently altered the results of the study. Eor example, items 
could have been worded to link more directly technology use and each of the five clusters of 
educationally effective practices. Questions could be asked about whether students receive 
support from services available through a campus website (Supportive Campus Environment) or 
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about whether students participated in a course or program to learn a new computer technology 
(Enriching Educational Experiences). Nonetheless, that a reliable scale was derived from the 
technology items indicates that some meaningful aspects of the student experience are being 
captured. Moreover, we were judicious in our selection of items for the factor analysis with core 
survey items and technology items and keep the conclusions drawn from this analysis bounded 
by both the limitations and the exploratory nature of the investigation. 

Discussion and Implications 

Many campuses have invested substantial amounts of resources to make various forms of 
information technology accessible to students. The relatively high frequency with which 
students are using information technology for educational purposes and the positive relationships 
between students’ academic uses of information technology and multiple aspects of student 
engagement suggest that those investments are paying off. The findings from this study and 
others (e.g., Kuh & Hu, 2001; Twigg, 2004) indicate using information technology is associated 
with desirable outcomes. At the same time, information technology introduces opportunities for 
mischief, such as cyber plagiarism — students’ inappropriate use of material taken without 
attribution from online sources. Reisberg (2000) catalogues other worrisome side effects of 
overuse of information technology that are worthy of further investigation. 

The relative strength of the positive relationships between academic uses of information 
technology and engagement, particularly academic challenge, student-faculty interaction, and 
active and collaborative learning, suggest that, at the very least, there is overlap between these 
constructs. That both technology-related and non-technology-related collaborative learning 
items factor together and produce a reliable scale, suggests that areas of engagement like student- 
faculty interaction and collaborative learning may well be related to information technology. 
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That is, students who frequently use information technology are likely to have more frequent 
interactions with faculty members. 

Asking students about their technology use may produce more reliable measures of other 
forms of engagement in effective educational practice. This seems to be the case, for example, 
for collaborative learning. Should future analyses show that in addition to being reliable, such 
measures are also better predictors of student outcomes, then it may make sense to modify 
certain questions aimed to measure various dimensions of student engagement. That said, there 
is also the possibility that such combined measures will not perform as well as predictors of other 
behaviors of interest. Kuh and Hu (2001) found that the effects of using information technology 
on several outcome measures were quite small when an overall measure of student engagement 
was in the model. This “mediation” of the effect of engagement with information technology 
may result from both measures explaining the same variance in the dependent variables, an 
indication that engagement with information technology may simply be an additional measure of 
students’ overall engagement. Obviously, further investigation is needed. 

Conclusion 

With students using information technology on a daily basis for their academic and non- 
academic pursuits, there is a need to understand the educational effects of this use. The results of 
this study suggest that using information technology for educational purposes is linked to how 
today’s college students engage in effective educational practices (e.g., active and collaborative 
learning) more generally. 

As researchers and practitioners in the field of higher education, we are faced with choices 
on how to measure and conceptualize students’ engagement with information technology. Is 
information technology an avenue for engagement in other areas, is it its own form of 
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engagement, or is it possibly both of these? The results of this study prompt us to consider how 
established indicators of student engagement may benefit from tying information technology 
items to activities related to collaborative learning, for example. In doing so, however, two 
things should be kept in mind. First, measuring students’ engagement in information technology 
may not add to our ability to explain educational outcomes above and beyond what is already 
captured by other measures of student engagement. If this is the case, we need to ask whether 
value is added by their inclusion. Second, we should not be boxed in by what we already 
measure. It is important to ask if there are ways students engage information technology that are 
independent of the established indicators of engagement represented by the NSSE survey and 
other instruments. 
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Table 1. 

Experimental Information Technology Items Used on the Online Version of NSSE 2003 

Item Wording 

1 How often do your instructors require you to use information technology (WWW, 
internet, computer conferencing, online testing, multimedia, etc.), other than word 
processing, to complete course assignments? 

2a Used computer and information technology when making class presentations? 

2b Communicated with classmates online to complete academic work? 

2c Worked in teams during class using information technology? 

2d Worked in teams outside of class using information technology to complete course 
assignments? 

2e Used email to ask an instructor to clarify an assignment? 

2f Expressed ideas to a professor via email that you did not feel comfortable saying in class? 

2g Used your institution’s library website to obtain resources for your academic work? 

2h Used another library (local library, another institution’s library) website to obtain 
resources for your academic work? 

2i Asked a librarian at your school for help in obtaining resources for your academic work? 

2j Used the WWW to obtain resources for your academic work? 

2k Made judgments about the quality of information you find on the WWW for use in your 

academic work? 

3 How often do your instructors use information technology in the classroom? 

4 How many courses are you taking this semester that are offered entirely online via the 
WWW, internet or email?^* 

5 To what extent do you gain new insights into course material from online discussions?’’ 

6a. Spending time online (WWW/internet/email) for any reason?^’ 

6b Spending time online (WWW/intemet/email) doing academic work?‘’ 

7 How often do you think other students at your institution copy and paste information from 
the WWW or internet into reports and papers without citing the source? 

Note: Except where noted, variables were measured on a 4-point scale (l=Never, 2=Sometimes, 3=Often, 4=Very Often) 

“ Responses range from 1=0 to 5=4 or more 
Responses for this item were l=Very little, 2=Some, 3=Quite a bit, 4=Very much 
Responses for this item were 1=0, 2=1-5, 3=6-10, 4=11-15, 5=16-20, 6=21-25, 7=26-30, 8=More than 30 
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Table 2. 

Reliabilities of the Clusters of Effective Educational Practices (Student- Level Scales) 



Scale Cronbach’s a 

Academic Challenge (1 1-item scale) 0.71 

Active and Collaborative Learning (7-item scale) 0.63 

Student-Faculty Interaction (6-item scale) 0.76 

Enriching Educational Experiences (12-item scale) 0.56 

Supportive Campus Environment (6-item scale) 0.76 



Note: Component items for each scale are described in National Survey of Student Engagement, 2003 
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Table 3. 



Students’ Experiences with Information Technology 



Variable 


First- Year 
Students 


Seniors 


InsXuxclox^ frequently (often or very often) require you to use information 
technology (WWW, internet, computer conferencing, online testing, 
multimedia, etc.), other than word processing, to complete course assignments 


78% 


81% 


Frequently used computer and information technology when making class 
presentations 


56% 


75% 


Frequently communicated with classmates online to complete academic work 


51% 


58% 


Frequently worked in teams during class using information technology 


31% 


36% 


Frequently worked in teams outside of class using information technology to 
complete course assignments 


41% 


58% 


Frequently used email to ask an instructor to clarify an assignment? 


58% 


79% 


Frequently expressed ideas to a professor via email that you did not feel 
comfortable saying in class 


26% 


29% 


Frequently used your institution’s library website to obtain resources for your 
academic work 


55% 


64% 


Frequently used another library (local library, another institution’s library) 
website to obtain resources for your academic work 


18% 


27% 


Frequently asked a librarian at your school for help in obtaining resources for 
your academic work 


24% 


26% 


Frequently used the WWW to obtain resources for your academic work? 


80% 


86% 


Frequently made judgments about the quality of information you find on the 
WWW for use in your academic work 


71% 


79% 


Instructors frequently use information technology in the classroom 


65% 


69% 


Taking 1 or more courses this semester that are offered entirely online via the 
WWW, internet or email 


11% 


10% 


Gain new insights into course material from online discussions quite a bit or 
very much 


35% 


33% 


Spend greater than 5 hours per week online (WWW/intemet/email) for any 
reason? 


73% 


69% 


Spend greater than 5 hours per week online (WWW/internet/email) doing 
academic work? 


38% 


39% 


Think other students at your institution frequently copy and paste information 
from the WWW or internet into reports and papers without citing the source? 


25% 


31% 






Engagement with Information Technology 27 



Table 4. 



Factor Loadings for Technology Items 



Factor Solutions 






Three 




Two 




One 


Item wording 


FI 


F2 


F3 


FI 


F2 


FI 


Worked in teams outside class using 
information technology 


0.77 






0.81 




0.74 


Worked in teams during class using 
information technology 


0.76 






0.80 




0.67 


How often do your instructors use information 
technology in the classroom 


0.69 






0.69 




0.58 


Communicated with classmates online to 
complete academic work 


0.68 






0.72 




0.71 


How often do your instructors require you to 
use information technology 


0.67 






0.67 




0.64 


Used computer and information technology 
when making class presentations 


0.67 






0.69 




0.67 


Used another library website to obtain 
resources for your academic work 




0.73 






0.63 


0.38 


Expressed ideas to a professor via email that 
you did not feel comfortable saying in class 




0.66 




0.31 


0.34 


0.54 


Used your institution's library website to 
obtain resources for your academic work 




0.62 


0.36 




0.76 


0.41 


Used email to ask an instructor to clarify an 
assignment 


0.34 


0.53 




0.40 


0.38 


0.66 


Made judgements about the quality of 
information you find on the WWW 






0.81 




0.71 


0.53 


Used the WWW to obtain resources for your 
academic work 






0.79 




0.60 


0.57 


% var. explained 


36.21 


10.96 


9.46 


36.21 


10.96 


36.21 


Cronbach’s alpha^* 


0.82 


0.63 


0.72 


0.83 


0.63 


0.83 



Note: Factor analyses performed using Principle Components extraction with Promax (oblique) rotation (N=63,540). 

^ Cronbach's alpha calculated using those items that load over 0.30 on a factor. Items that load on two factors were included in 
alpha calculations only for the factor on which they load the highest. 
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Table 5. 

Partial Correlations between Engagement Scales (N=62,586) 



Scales 


1 


2 


3 


4 


5 


6 


1 . Engagement with IT 


1.00 












2. Engagement with IT- Short 


0.86 


1.00 










3. Academic Challenge 


0.42 


0.33 


1.00 








4. Active & Collaborative Eearning 


0.47 


0.45 


0.47 


1.00 






5. Student-Eaculty Interaction 


0.42 


0.32 


0.46 


0.54 


1.00 




6. Enriching Educational Experiences 


0.33 


0.26 


0.38 


0.42 


0.48 


1.00 


7. Supportive Campus Environment 


0.31 


0.27 


0.34 


0.32 


0.39 


0.31 



Note; Partial correlations calculated controlling for gender, race, age, parents' education, class (first-year or 
senior), transfer status, living on campus, social fraternity/sorority membership, participation in athletics, 
full-time/part-time status, and major. 

Note: Correlations between scales nearly identical (differences less than 0.05) to partial correlations. 
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Table 6. 

Factor Loadings for Core Survey Items (Ql) and Select Technology Items 

Analysis on Ql Analysis with Tech Items 



Item wording 


FI F2 


F3 


F4 


FI 


Talked about career plans with a faculty member or 
advisor 


0.79 






0.81 


Discussed ideas from your readings or classes with 

faculty members outside of class 

Worked with faculty members on activities other 


0.77 






0.79 


than coursework 


0.77 






0.80 


Discussed grades or assignments with an instructor 
Received prompt feedback from faculty on your 


0.57 






0.60 


academic performance 


0.56 






0.56 


Tutored or taught other students (paid or voluntary) 
Asked questions in class or contributed to class 


0.51 






0.55 


discussions 


0.49 






0.58 


Participated in a community-based project as part 
of a regular course 


0.30 






0.39 



Worked with other students on projects during class 
Worked with classmates outside of class to prepare 




0.73 








0.40 






class assignments 




0.70 








0.57 






Made a class presentation 

Used an electronic medium to discuss or complete 




0.54 








0.32 






an assignment 

Put together ideas or concepts from different 




0.49 








0.45 






courses when completing assignments 




0.39 






0.34 








Used e-mail to communicate with an instructor 


0.34 


0.34 






0.34 


0.30 






Had serious conversations with students who are 

very different from you 

Had serious conversations with students of a 






0.91 








0.90 




different race or ethnicity than your own 
Included diverse perspectives in class discussions 






0.91 








0.90 




or writing assignments 

Discussed ideas from your readings or classes with 






0.42 


0.41 






0.40 


0.45 


others outside of class 


0.35 




0.39 




0.38 




0.37 




Prepared two or more drafts of a paper or 

assignment before turning it in 

Come to class without completing readings or 








0.77 








0.79 


assignments 

Worked on a paper or project that required 




0.36 




-0.69 








-0.60 


integrating ideas from various sources 

Worked harder than you thought you could to meet 




0.34 




0.53 








0.60 


an instructor's standards 








0.45 








0.43 



Technology Items 

Used computer and information technology when 

making class presentations 0.7 1 

Communicated with classmates online to complete Variables not included in this 

academic work analysis 0.76 

Worked in teams during class using information 

technology 0.79 

Worked in teams outside class using information 

technology 0.85 

% var. explained 26.23 6.87 6.01 5.64 25.27 8.26 5.71 5.00 

Cronbach's alpha"* 0.78 0.65 0.70 0.57 0.84 0.79 0.79 0.62 



Note: Factor analyses performed using Principle Components extraction with Promax (oblique) rotation (n = 63,407). 

^ Cronbach's alpha calculated using those items that load over 0.30 on a factor. Items that load on two factors were included in alpha calculations 
for the factor on which they load the highest. 




